Language models have seen enormous progress on advanced benchmarks in recent years, but much of this progress has only been possible by using more costly models. Benchmarks may therefore present a warped picture of progress in practical capabilities per dollar. To remedy this, we use data from Artificial Analysis and Epoch AI to form the largest dataset of current and historical prices to run benchmarks to date. We find that the price for a given level of benchmark performance has decreased remarkably fast, around $5\times$ to $10\times$ per year, for frontier models on knowledge, reasoning, math, and software engineering benchmarks. These reductions in the cost of AI inference are due to economic forces, hardware efficiency improvements, and algorithmic efficiency improvements. Isolating out open models to control for competition effects and dividing by hardware price declines, we estimate that algorithmic efficiency progress is around $3\times$ per year. Finally, we recommend that evaluators both publicize and take into account the price of benchmarking as an essential part of measuring the real-world impact of AI.

OpenAI is taking on Mark Zuckerberg's Meta and Chinese rival DeepSeek by launching its own freely available artificial intelligence models. The ChatGPT developer has announced two "open weight" large language models, which are free to download and can be customised by developers. Meta's Llama models are available on a similar basis, and OpenAI's move marks a departure from ChatGPT, which is based on a "closed" model that cannot be customised. Sam Altman, OpenAI's chief executive, said the company was excited to add to a stack of freely available AI models "based on democratic values … and for wide benefit". He added: "We're excited to make this model, the result of billions of dollars of research, available to the world to get AI into the hands of the most people possible." OpenAI said the models could underpin an AI agent that operates autonomously, and that they were "designed to be used within agentic workflows".

Sam Altman today revealed that OpenAI will release an open weight artificial intelligence model in the coming months. "We are excited to release a powerful new open-weight language model with reasoning in the coming months," Altman wrote on X. Altman said in the post that the company has been thinking about releasing an open weight model for some time, adding "now it feels important to do." The move is partly a response to the runaway success of the R1 model from Chinese company DeepSeek, as well as the popularity of Meta's Llama models. OpenAI may also feel the need to show that it can train the new model more cheaply, since DeepSeek's model was purportedly trained at a fraction of the cost of most large AI models. "This is amazing news," Clement Delangue, cofounder and CEO of HuggingFace, a company that specializes in hosting open AI models, told WIRED.